Skip to content

Add ability to use convert_query_response with Python Transforms #376

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 29, 2025
Merged

Add ability to use convert_query_response with Python Transforms #376

merged 1 commit into from
Apr 29, 2025

Conversation

@ogenstad
Copy link
Contributor

@ogenstad ogenstad commented Apr 24, 2025 •
edited
Loading

This PR introduced support for "convert_query_response" for Python Transforms.

Changes:

  • The client method to clone the config has been moved to the config object itself and modified to not only rely on deepcopy. The reason for this comes from the tests within the Infrahub directory where we've overridden the client transport to a method that doesn't allow for pickling. As the transport itself shouldn't be of interest for the config we reuse some of the elements in the cloned config
  • As some of the methods and logic is shared between Generators and Transforms a new "InfrahubOperation" class has been introduced and logic has been transfered and consolidated from both Generators and Transforms to this new base class.
  • Some new protocols were added
  • I added some new integration tests for both Transforms and Generators, previously these weren't tested within the SDK but and instead relied on tests within the Infrahub repository, but added to the SDK tests to have better coverage
  • I ran into some issues with the tests for the "infrahubctl generator" typer app as it's an async app, after some troubleshooting I noticed that the typer.testing.CliRunner doesn't seem to support tests for async commands like this. Instead of spending more time on that I opted to call the generator function directly within the code for now. It would be good with a permanent solution for that but I didn't want to spend more time on it now.

Fixes #281

Backend PR: opsmill/infrahub#6363

@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Apr 24, 2025 •
edited
Loading

Deploying infrahub-sdk-python with  Cloudflare Pages  Cloudflare Pages

Latest commit: cfbb217
Status: ✅  Deploy successful!
Preview URL: https://b865e123.infrahub-sdk-python.pages.dev
Branch Preview URL: https://pog-transform-convert-query.infrahub-sdk-python.pages.dev

View logs

@codecov
Copy link

codecov bot commented Apr 24, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 93.15068% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
infrahub_sdk/operation.py 88.63% 2 Missing and 3 partials ⚠️ 
@@             Coverage Diff             @@
##           develop     #376      +/-   ##
===========================================
+ Coverage    73.94%   75.12%   +1.18%     
===========================================
  Files           92       93       +1     
  Lines         8540     8560      +20     
  Branches      1676     1678       +2     
===========================================
+ Hits          6315     6431     +116     
+ Misses        1779     1676     -103     
- Partials       446      453       +7
Flag Coverage Δ
integration-tests 34.42% <61.64%> (+9.00%) ⬆️
python-3.10 47.52% <47.94%> (+0.13%) ⬆️
python-3.11 47.50% <47.94%> (+0.11%) ⬆️
python-3.12 47.50% <47.94%> (+0.13%) ⬆️
python-3.13 47.52% <47.94%> (+0.13%) ⬆️
python-3.9 45.82% <23.28%> (+0.04%) ⬆️
python-filler-3.12 24.26% <30.13%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
infrahub_sdk/client.py 69.79% <100.00%> (+1.75%) ⬆️
infrahub_sdk/config.py 88.28% <100.00%> (+0.90%) ⬆️
infrahub_sdk/ctl/cli_commands.py 69.13% <100.00%> (+0.54%) ⬆️
infrahub_sdk/ctl/generator.py 51.78% <ø> (+28.57%) ⬆️
infrahub_sdk/generator.py 77.50% <100.00%> (+41.60%) ⬆️
infrahub_sdk/protocols.py 100.00% <100.00%> (ø)
infrahub_sdk/schema/repository.py 86.33% <100.00%> (+2.99%) ⬆️
infrahub_sdk/transforms.py 74.28% <100.00%> (+4.07%) ⬆️
infrahub_sdk/operation.py 88.63% <88.63%> (ø)

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ogenstad ogenstad force-pushed the pog-transform-convert-query-response-IHS-105 branch 2 times, most recently from 3f7f947 to bdc0d4a Compare April 28, 2025 11:24
@ogenstad ogenstad mentioned this pull request Apr 28, 2025
Merged
@ogenstad ogenstad requested a review from a team April 28, 2025 13:28
ajtmccarty
ajtmccarty approved these changes Apr 28, 2025
View reviewed changes
infrahub_sdk/ctl/cli_commands.py
branch=branch,
infrahub_node=InfrahubNode,
convert_query_response=transform_config.convert_query_response,
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you think it is worth passing the whole transform_config here instead of just convert_query_response? might make it easier to add other parameters like this in the future

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I do, but it would require more changes in the backend when we use this outside of the CTL in situations where we haven't stored the config anywhere. I think for now we should leave it like this to not change too many things. But could be that we should do a larger refactor and consider what parts should be kept in the database and what options should only exist in Git.

@ogenstad ogenstad force-pushed the pog-transform-convert-query-response-IHS-105 branch from bdc0d4a to cfbb217 Compare April 29, 2025 07:23
@ogenstad ogenstad merged commit 5e3e379 into develop Apr 29, 2025
18 checks passed
@ogenstad ogenstad deleted the pog-transform-convert-query-response-IHS-105 branch April 29, 2025 13:48
@ogenstad ogenstad mentioned this pull request Apr 29, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ajtmccarty ajtmccarty ajtmccarty approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ogenstad @ajtmccarty